The invention relates to a parallax image pickup apparatus and an image pickup method which are applied to creation of, for example, a holographic stereogram.
Two-dimensional images of an object which are obtained by seeing it from different visual points are used as original images and holograms to reproduce a three-dimensional image can be synthesized. A holographic stereogram is formed by, for example, a method whereby a number of images obtained by sequentially photographing an object from different observing points are used as original images and they are sequentially recorded as strip-shaped or dot-shaped element holograms onto a sheet of recording medium for a hologram.
For example, as for a holographic stereogram having parallax information only in the lateral direction, as shown in FIG. 10, a plurality of parallax images 12a to 12e are sequentially obtained by sequentially photographing an object 11 from different observing points in the lateral direction. Each image of a parallax image train (12a to 12e) is arranged so that, for example, a line focal point is formed on recording medium 13 for a hologram and those parallax images are recorded thereon at a proper angle by using reference light. That is, as shown in FIG. 11, the image is divided in a strip shape in the parallax direction and strip-shaped data in the different parallax images is reconstructed, that is, an image process of what is called “slice and Dice” is executed, thereby forming a hologram image D2. The hologram image D2 is formed by sequentially recording the strip-shaped element holograms onto the recording medium 13 for the hologram by an image recording apparatus.
In the holographic stereogram, since the image information obtained by sequentially photographing the object from the different observing points in the lateral direction has sequentially been recorded in the lateral direction as strip-shaped element holograms, when such a holographic stereogram is seen by both eyes of the observer, 2-dimensional images projected to his right and left eyes slightly differ. Thus, the observer feels a parallax and a 3-dimensional image is reproduced. The 3-dimensional image as well as the holographic stereogram is simply and properly referred to as a “hologram” hereinbelow.
As an image having a display effect similar to that of the holographic stereogram, there is also a stereogram using a microlens array such as lenticular or the like. Although the number of parallax images which can be recorded is generally smaller than that of the holographic stereogram, they are analogous in terms of a point that video images obtained by sequentially photographing an object from different observing points are recorded.
According to the parallax image train pickup apparatus to obtain such a plurality of original images, the object is photographed by a camera unit such as mobile camera, multiple-eye camera, or the like and a number of photographed images including the parallax information are formed. FIGS. 12A to 12D show a plurality of examples of a layout relation between an object 31 and a camera unit 32 in the parallax image train pickup apparatus.
The example shown in FIG. 12A relates to a form in which one camera unit 32 is horizontally moved for a predetermined time with respect to the object 31, the direction of the camera unit 32 to the object 31 is not changed, and the direction in which the camera unit 32 targets the object is not changed (what is called “Straight-ahead Camera on Straight Track”). That is, according to such a form, the camera unit 32 photographs the object 31 without typically targeting it.
FIG. 12B shows a form in which one camera unit 32 is horizontally moved for a predetermined time with respect to the object 31, the direction of the camera unit 32 to the object 31 is changed, and the direction in which the camera unit 32 targets the object is not changed (what is called “Panning Camera on Straight Track”). That is, according to such a form, since the direction of the camera unit 32 to the object 31 is changed in association with the horizontal movement of the camera unit 32, the camera unit 32 typically targets the object 31 and photographs it.
FIG. 12C shows a form in which one camera unit 32 is horizontally moved for a predetermined time with respect to the object 31, the direction of the camera unit 32 to the object 31 is not changed, and the direction in which the camera unit 32 targets the object is changed (what is called “Re-centering Camera on Straight Track”). That is, according to such a form, although the direction of the camera unit 32 to the object 31 is not changed in association with the horizontal movement of the camera unit 32, since an image pickup device and the like in the camera unit 32 are moved, the camera unit 32 typically targets the object 31 and photographs it.
The parallax image train pickup apparatus having the layout shown in FIG. 12C has been disclosed in JP-A-2000-066568. According to such an apparatus, although the direction of a photographing lens is not changed, a CCD camera is horizontally moved synchronously with that the camera unit is moved in the horizontal direction by a stepping motor, even if the camera unit is moved in the horizontal direction, the CCD camera can target the object at the center through the photographing lens.
Further, FIG. 12D shows a form in which one camera unit 32 is moved on a surrounding circumference of the object 31 for a predetermined time, (what is called “Rotating Camera”). The form in which the object 31 is rotated is called “Rotating Object”. According to such a form, the camera unit 32 is circularly moved around the object 31, typically targets the object 31, and photographs it.
In the parallax image train pickup apparatus, it is also possible to provide a plurality of camera units 32 and simultaneously photograph the object 31, or the object 31 may be moved instead of the camera unit 32.
The parallax image train pickup apparatuses in the four forms shown in FIGS. 12A to 12D have a merit and a demerit, respectively. According to the form of “Straight Track” shown in FIG. 12A, although its structure is simple, since it is necessary to photograph the object at a wide angle of view so that the object is sufficiently included and most of the photographed image except for the object image is not used, valid resolution deteriorates.
According to the Re-centering form shown in FIG. 12C, it is necessary to form an image onto an area whose area is wider than that of the image pickup device and whose aspect ratio differs from the normal aspect ratio in accordance with the image pickup device or form anamorphic images of different vertical/lateral image forming magnifications. Anyway, it is necessary to manufacture a special optical lens system. Although a commercially available lens such as a lens for 35 mm of a silver salt photograph or the like can be also used, in such a case, there is such a limitation that if the image pickup device is too large contrarily, it is difficult to obtain high resolution, and if it is too small, it is difficult to obtain a wide angle of view. A possibility that the commercially available image pickup device can be used as it is is small.
According to the form (what is called Panning Camera on Straight Track) shown in FIG. 12B and the form (what is called Rotating Camera) shown in FIG. 12D, since the inherent camera unit can be used in its original form, there is an advantage that the apparatus can be constructed without remodeling the latest camera of high quality.
According to the form shown in FIG. 12D, since the object is fixed and the camera is moved along a circular orbit or since the camera is fixed and the object is rotated, there is an advantage that the object can be easily photographed by a relatively simple construction. However, in the case of forming a plane hologram, the following drawbacks can be mentioned: it is necessary to correct a trapezoidal distortion; and more strictly speaking, a distortion which cannot be fully corrected remains later and it is difficult to obtain accurate 3-dimensional localization information. Although the former trapezoidal distortion can be corrected by image processes, the latter is a perspective distortion which is caused in the case where although only the data obtained by photographing the object from the predetermined positions of the camera and the object is obtained, when the plane hologram is recorded, the image has to be constructed on the basis of information in which a distance between the hologram surface and the object is changed, that is, it is an uncorrectable distortion. For example, when the distance is fixed to 1 m although the object has to be photographed from the position which is away from the camera by 1.4 m, an unnatural distortion occurs.
In respect of this point, the form shown in FIG. 12B has an advantage that an image which has no perspective distortion and is suitable to form the plane hologram is obtained except for a distortion or the like of the optical lens system. On the other hand, since the distance between the camera and the object changes, there is a drawback in which it is difficult to obtain an accurate focal point.
Similarly, if the distance between the camera and the object changes, in the case of the same angle of view, a size of object changes. In the case of forming the plane hologram, even if a principle visual point position changes, the object needs to be seen in the same size. Therefore, a zooming process is necessary. Although such a zooming process can be also executed by an image process after the photographing, since the unnecessary zoomed image which is not used is photographed, there is a problem of deterioration of the resolution. In the case of monitoring the image in a real-time manner, there is also such a problem that unnaturalness in which the size of object changes remains.
Explanation will be supplemented with respect to the case of monitoring the image in a real-time manner. In FIG. 13, reference numeral 50 denotes a parallax image train pickup apparatus. In the parallax image train pickup apparatus 50, a half mirror 40 is arranged between the object 31 and the camera unit 32. That is, the camera unit 32 is arranged in a space surrounded by an upper substrate 42, a lower substrate 41, a side wall 43, and the half mirror 40.
A photographing unit 33 in the parallax image train pickup apparatus 50 has: the camera unit 32; a camera unit feeding mechanism 34, and an illuminating light source 35a for illuminating the object 31 which is being photographed.
The camera unit 32 has: for example, a ⅔-inch CCD camera 36 for photographing; the camera unit feeding mechanism 34 which is driven by a stepping motor; and a photographing lens 39. The camera unit 32 is put on the camera unit feeding mechanism 34 having a predetermined length, for example, a whole length of 2700 mm. When the photographing is started, the camera unit feeding mechanism 34 is driven by the stepping motor (not shown) or the like. The camera unit 32 is moved in the horizontal direction by the driving of the camera unit feeding mechanism 34.
In this case, in the parallax image train pickup apparatus 50, since the camera unit 32 is shut out by the half mirror 40, the object 31 hardly see the camera unit 32. Therefore, in the parallax image train pickup apparatus, the object 31 is not conscious of the movement of the camera unit 32 and a number of pickup images including parallax information can be photographed.
A television monitor 51 is arranged on the upper substrate 42. For example, an image obtained by photographing the object 31 from the front surface thereof is displayed together with the following conditions necessary for the photographing. That is, such conditions that information indicative of the photographing position of the object 31 such as image frames of the holographic stereogram which is formed and another image to be synthesized with the photographed images can be recognized by the object 31 prior to photographing.
It is possible to move the CCD camera 36 to the front position of the object 31 prior to photographing and photograph the image of the object 31 or it is also possible to arrange another CCD camera 52 or the like to the position in front of the object 31 and photograph it.
The image which is displayed on the TV monitor 51 while a pickup image D1 is being photographed may be a still image photographed before the photographing as mentioned above or the image from the front side of the object 31 during the photographing of the pickup image may be also displayed.
Instead of the image photographed from the front side of the object 31 as mentioned above, a number of pickup images including the parallax information of the object 31 which is photographed may be sequentially displayed.
By constructing as mentioned above, in the parallax image train pickup apparatus 50, the object 31 is not conscious of the movement of the camera unit 32 and a number of pickup images including the parallax information can be photographed. By allowing the object 31 to observe its own image during the photographing, the sight line of the object 31 is fixed and the object 31 is not unnecessarily moved. Therefore, the parallax image train pickup apparatus 50 can form the pickup image of high picture quality.
In the parallax image train pickup apparatus 50, the object 31 can grasp the conditions necessary for the photographing such as its own photographing position and the photographing state before and during the photographing.
Further, when the holographic stereogram is formed by synthesizing with another image, the parallax image train pickup apparatus 50 can presume the completed holographic stereogram.
As another construction, by arranging the monitor onto camera rails in a height position almost near the camera on the side opposite to the object without using the half mirror 40, it is also possible to enable the object to see the monitor during the photographing.
According to the form shown in FIG. 12B, in the case of moving the camera unit 32 on the straight camera rails, at the beginning and the last of the photographing when the camera unit 32 is located near both ends of the rails, the object is photographed from a remote position as compared with the state where the camera unit 32 is located at the center. Therefore, information which is not printed is photographed to a peripheral portion of the completed hologram. In the case of using such a real-time monitor, there is a possibility that the framing becomes erroneous. Therefore, it is important that the size of object is not changed and information close to the framing of the completed hologram can be monitored.
Since the camera is rotated while being moved on the straight rails, it is necessary to control at least the operations of two axes and it is complicated.